chore(inputs.elasticsearch_query): Migrate to official ES client by alespour · Pull Request #19288 · influxdata/telegraf

alespour · 2026-07-15T16:43:04Z

Summary

Refactors plugin code to use official ES client instead of unmaintained olivere/elastic.
Adds ES 5 integration test coverage.

Node discovery caveats

As results of official clients limitations:

ES 5 configurations using enable_sniffer = true will log warning because the official v5 client does not support node discovery, and continue.
ES 6 client in-flight node discovery cannot be cancelled because DiscoverNodes() has no context support. Shutdown waits for it to finish, bounded by the configured HTTP timeout (5s by default).
For ES 6, health_check_interval controls how often nodes are discovered when enable_sniffer = true; olivere-style health checks are no longer performed.

Checklist

No AI generated code was used in this PR
AI generated code used in this PR follows the InfluxData Policy on AI-Generated Code Contributions

Related issues

resolves #19212

srebhan

Thanks @alespour for your work! I have some comments...

srebhan · 2026-07-17T10:15:22Z

@alespour regarding the question if we need the v5 client if code works with v6, I'd say no. If you can cover multiple ES server versions with one client, go for it as long as this doesn't limit the queries or anything...

alespour · 2026-07-17T11:23:04Z

@srebhan Thank you very much for very thorough review. I'm overhauling the PR.

alespour · 2026-07-20T09:37:40Z

@srebhan PR is overhauled. Plugin uses concrete client implementation for each major version (v5, v6). Version-independent logic is now in shared functions (code blocks moved client_v5.go -> client.go, without logic change).

One thing that requires maintainers decision: v5 client does not support discovery. Now, when enable_sniffer = true and v5 backend detected, plugin fails to start with an error. This is a breaking change.

I can think of 3 solutions:

a) document it as breaking change
b) log warning and silently ignore
c) use v6 client, but, there is a but - although all tests (unit and integration) using v6 client against ES v5 backend pass OK, there is no official guarantee about v6 client compatibility with older ES versions. Just empirical validation with tests. REST interface seems stable between v5 and v6 and official client is fairly low-level, so the fact that tests pass is fairly predictable.

srebhan · 2026-07-21T16:05:45Z

Thanks @alespour for the update, I will review in a minute...

Regarding

I can think of 3 solutions:

Please issue a warning and silently ignore discovery for v5. I think this is best as it is unlikely someone is still on 5.x and uses discovery. Maybe also add a short comment in the README.md that v5 does not support discovery.

srebhan

Nice update @alespour! Some more comments from my side...

srebhan · 2026-07-21T16:16:18Z

+		if interval <= 0 {
+			return
+		}


Can we please check this outside of this function?!

addressed in 6b817e4

srebhan · 2026-07-21T16:20:25Z

+	ctx, cancel := context.WithCancel(context.Background())
+
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {


I would prefer to do the goroutine setup on the caller side and pass the context in here. This saves the returned function and makes it clear that the client runs another thread...

addressed in 6b817e4

srebhan · 2026-07-21T16:21:01Z

+
+// startDiscovery runs node discovery immediately and repeats it when the interval is positive.
+// The returned function stops the loop and waits for any active call to return.
+func startDiscovery(log telegraf.Logger, interval time.Duration, discover func(context.Context) error) func() {


Can you please make log the last argument?! We usually do this in other places...

addressed in 6b817e4

srebhan · 2026-07-21T16:24:05Z

+	if cfg.enableSniffer {
+		cfg.httpClient.CloseIdleConnections()
+		return nil, errors.New("enable_sniffer is not supported by the official ElasticSearch v5 client")
 	}


Can we please check this on the caller side?

fixed in 89f4a0d

srebhan · 2026-07-21T16:30:20Z

 	}
+}
+
+func (e *ElasticsearchQuery) Start(telegraf.Accumulator) error {


Can we please merge this with newClient?!

fixed in dc8ae3d

srebhan

This looks very nice and clean @alespour! Thank you very much. I have two more comments to make this perfect. ;-)

srebhan

Thanks a lot @alespour!

skartikey

@alespour Thanks! I have a couple of question/comment, please take a look.

skartikey · 2026-07-23T13:53:52Z

 	mapMetricFields map[string]string
 	measurements    map[string]map[string]string
-	queries         interface{} // client specific data to execute the query
+	queries         interface{} // prepared once and reused across collections


[nit] queries is interface{} but it's only ever []queryData, which forces the unchecked assertion at client.go:246 (and the comment there rationalizing the panic). Same shape at client.go:399 where query()'s interface{} return gets asserted back to map[string]json.RawMessage. Both are internal round-trips through the same package, so typing the field as []queryData and the client.query return as map[string]json.RawMessage drops two assertions and the "programming error" caveat. Happy to be argued out of it if the interface{} is deliberate for a later ES7 shape.

No, interface{} is not needed V7+, with near certainty.

telegraf-tiger · 2026-07-23T15:19:17Z

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

⚠️ This pull request increases the Telegraf binary size by 1.94 % for linux amd64 (new size: 314.2 MB, nightly size 308.2 MB)

📦 Click here to get additional PR build artifacts

Artifact URLs

. DEB	. RPM	. TAR . GZ	. ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

skartikey

@alespour Thanks for the contribution!

fix: migrate plugin to official ES clients

04036dc

telegraf-tiger Bot added fix pr to fix corresponding bug plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins labels Jul 15, 2026

alespour added 2 commits July 16, 2026 11:39

chore: update dependencies licenses file

19d43c9

fix: preserve error details

7c1dd27

alespour changed the title ~~fix(inputs.elasticsearch_query): migrate plugin code to official ES clients~~ fix(inputs.elasticsearch_query): migrate plugin code to official clients Jul 16, 2026

alespour added 2 commits July 16, 2026 12:45

fix: improve code comment

8f763fe

chore: trigger CI workflow

059f78e

alespour marked this pull request as ready for review July 16, 2026 12:01

srebhan changed the title ~~fix(inputs.elasticsearch_query): migrate plugin code to official clients~~ chore(inputs.elasticsearch_query): Migrate to official ES client Jul 16, 2026

telegraf-tiger Bot added the chore label Jul 16, 2026

srebhan reviewed Jul 16, 2026

View reviewed changes

srebhan self-assigned this Jul 16, 2026

srebhan added area/elasticsearch and removed fix pr to fix corresponding bug labels Jul 16, 2026

alespour added 4 commits July 17, 2026 15:30

refactor: simplify official client integration

e8f7e3a

fix: discovery

ac9d16e

test: add dicsovery test

da37742

fix: preserve server version error messages

b318ea4

alespour requested a review from srebhan July 20, 2026 15:07

srebhan reviewed Jul 21, 2026

View reviewed changes

alespour added 5 commits July 22, 2026 09:32

fix: remove unnecessary cast

f74f7ae

fix: accept only 2xx

2b9a46e

refactor: discovery lifecycle

6b817e4

refactor: extract API types

7559508

fix: ignore unsupported ES5 discovery

89f4a0d

alespour added 4 commits July 22, 2026 10:25

fix: add context to response decoding errors

5e336ef

refactor: merge client setup into Start

dc8ae3d

test: ignore error message prefixes

5c5e7c9

test: remove redundant error assertion

873ca77

alespour requested a review from srebhan July 22, 2026 15:17

srebhan reviewed Jul 22, 2026

View reviewed changes

Comment thread plugins/inputs/elasticsearch_query/client.go Outdated

Comment thread plugins/inputs/elasticsearch_query/client.go Outdated

refactor: make query builders methods on aggregation

e64857b

srebhan approved these changes Jul 23, 2026

View reviewed changes

srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Jul 23, 2026

srebhan assigned skartikey and unassigned srebhan Jul 23, 2026

skartikey reviewed Jul 23, 2026

View reviewed changes

fix: align node discovery interval and docs

a4dfdf4

skartikey approved these changes Jul 23, 2026

View reviewed changes

skartikey merged commit c66f373 into influxdata:master Jul 23, 2026
29 checks passed

github-actions Bot added this to the v1.39.3 milestone Jul 23, 2026

Uh oh!

Conversation

alespour commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Node discovery caveats

Checklist

Related issues

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

srebhan commented Jul 17, 2026

Uh oh!

alespour commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alespour commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

srebhan commented Jul 21, 2026

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

skartikey left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alespour Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

telegraf-tiger Bot commented Jul 23, 2026

alespour commented Jul 15, 2026 •

edited

Loading

alespour commented Jul 17, 2026 •

edited

Loading

alespour commented Jul 20, 2026 •

edited

Loading

alespour Jul 23, 2026 •

edited

Loading